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A self-control mechanism for the dynamics of a three-state fully-connected neural network is 
studied through the introduction of a time-dependent threshold. The self-adapting threshold is a 
function of both the neural and the pattern activity in the network. The time evolution of the order 
parameters is obtained on the basis of a recently developed dynamical recursive scheme. In the limit 
of low activity the mutual information is shown to be the relevant parameter in order to determine 
the retrieval quality. Due to self-control an improvement of this mutual information content as well 
as an increase of the storage capacity and an enlargement of the basins of attraction are found. 
These results are compared with numerical simulations. 
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I. INTRODUCTION 



It is well-known by now that low-activity neural net- 
work models have a larger storage capacity than the cor- 
responding models with a mean 50% activity (see, e.g., 
0). However, this improvement is not always apparent 
in the basins of attraction. Furthermore, for low activi- 
ties the information content in a single pattern is reduced. 
For these reasons it is argued that a neural activity con- 
trol system is needed in the dynamics of the network in 
order to keep its activity the same as the one for the 
memorized patterns during the whole retrieval process 
[g]. Recently, new suggestions have been put forward 
for the choice of threshold functions in network models 
in order to get an enhanced retrieval quality- overlap, 
basin of attraction, critical capacity, information con- 
tent (see ^-Q] and references therein). Diluted models 
H[H|] , layered models @ and models for sequential pat- 
terns Q have been considered. In all cases it has been 
found that appropriate thresholds lead to considerable 
improvements of the retrieval quality. 

The models mentioned above have a common property. 
For the diluted and layered models there is no feedback 
in the dynamics. For the model with sequential patterns 
no feedback correlations are taken into account. The ab- 
sence of feedback considerably simplifies the dynamics. 
Hence, it is interesting to look at a model with feed- 
back correlations and to see whether the introduction of 
a threshold in the sense described above still enhances the 
retrieval properties in this much more complex situation. 

With these ideas in mind we consider in the sequel 
low activity (or in other words sparsely coded) fully con- 
nected neural networks. In particular, we study the ap- 
plication of a self-control mechanism proposed recently 
for a diluted network of binary patterns fjl . Self-control 



has been introduced in order to avoid imposing some ex- 
ternal constraints on the network with the purpose of im- 
proving its retrieval properties. Such external constraints 
destroy the autonomous functioning of the network. 

The model we look at is a fully-connected attractor 
neural network with neurons and patterns taking the val- 
ues {— 1,0, +1} and pattern activity a. A low-activity 
neural network corresponds then to the case where the 
pattern distribution is far from uniform, i.e., a < 2/3. 
This network has the advantage that it can be generated 
keeping a symmetric distribution of the states since both 
the ±1 states are considered the active ones, while the 
state is the inactive one. 

The rest of this paper is organised as follows. The 
three-state network model and its order parameters are 
described in section II. In order to study the retrieval 
quality of the model, especially in the limit of low activity 
the mutual information content is analysed in Section III. 
Section IV discusses the dynamics of this network in the 
presence of the self-control mechanism realised through 
the introduction of a time-dependent threshold. Evolu- 
tion equations for the order parameters are written down. 
Using these equations the influence of self-control on the 
retrieval quality of the network - information content, 
critical capacity, basins of attraction - is studied in sec- 
tion V. Furthermore, these theoretical findings are com- 
pared with results from numerical simulations of a fully 
connected network of 10 neurons. Finally, section VI 
presents some concluding remarks. 



II. THE MODEL 

Consider a neural network model of N three-state 
neurons. At a discrete time step t the neurons <7j S 



{0, ±1}, i — 1,...,N are updated according to the par- 
allel deterministic dynamics 



°»,t+l — Fg t (hi t t), hi t t — Jij a j,t 



(1) 



where h i t is the local held of neuron i at time t and 
9 t a time-dependent threshold parameter. As usual, the 
transfer function Fg t is given by 



Fg t (x) = sgn{x)e(\x\ - t ) 



(2) 



with Q the standard Heaviside function. 

The couplings are determined as a function of the 
memorized patterns £f by the Hebbian learning algo- 
rithm 



p—aN 



(3) 



with a the loading capacity. The patterns are taken to 
be independent identically distributed random variables 
(IIDRV) $ e {0,±l},i = l,...,N,(j, = l,...,p, chosen 
according to the probability distribution 

p(o = a8(\m 2 -i) + (i-amn w 

with a = (|£f | 2 ) the activity of the patterns. More- 
over, we assume that there is no bias, i.e., (£f ) = and 
that there exist no correlation between patterns such that 

= o. 

At this point we remark that the long-time behavior of 
this network model is governed by the spin-1 Hamiltonian 



H = - JijViVj - #t £ °i 



(5) 



Furthermore, the Hopheld model can be recovered by 
taking the activity a — 1 and the threshold t = 0. 

The standard order parameters of this type of models 
are the retrieval overlap between the /ith-pattern and the 
microscopic state of the network 



N.t 



(6) 



and the neural activity of the neurons 

qN<t =lj2\^\ 2 - ( 7 ) 



In the next Section we use these order parameters in 
order to study the retrieval quality of the network. 



III. MUTUAL INFORMATION 

It is known that the Hamming distance between the 
state of the network and the pattern {£f }, viz. 



dt = -jy £ \& ~ CT M I 2 = a ~ 2am N,t 



QN.t 



(8) 



is a good measure for the retrieval quality of a network 
when the patterns are uniformly distributed, i.e., when 
the neural activity a — 2/3. But for low-activity net- 
works it cannot distinguish between a situation where 
most of the wrong neurons (cr, ^ £f ) are turned off and a 
situation where these wrong neurons are turned on. This 
distinction is critical in the low-activity three-state net- 
work because the inactive neurons carry less information 
than the active ones ||. Therefore the mutual informa- 
tion function I((Ti,t',€it) nas been introduced |||^] 

I(cr i<t ;^ t )=S(a itt )-(S(a i>t \^ t )) i , (9) 

where is considered as the input and cxi it as the out- 
put with Sfaij) its entropy and S^er^l^) its conditional 
entropy, viz. 



S((X it , 



•^p(cTi,t) bx\p(a i)t )} 



(10) 



(11) 



Here p(ai^) denotes the probability distribution for the 
neurons at time t and p(cr; it |£f t ) indicates the conditional 
probability that the i-th neuron is in a state cr^t at time 
t given that the z-th site of the stored pattern to be re- 
trieved is £f t . 

The calculation of the different terms of this mutual 
information for the model at hand proceeds as follows. 
As a consequence of the mean-field theory character of 
our model it is enough to consider the distribution of a 
single typical neuron so we forget about the index i in 
the sequel. We also do not write the time index t and 
the pattern index \i. 

The conditional probability that the ith neuron is in a 
state <ii at time t, given that the ith site of the pattern 
being retrieved is £j can be obtained as follows. Formally 
writing (O) = ((0) ale ) 6 = EfPtOE^MOO for an 
arbitrary quantity O and using the complete knowledge 
about the system (£) = 0, (a) — 0, (cr£) — am, (£ 2 ) = 



a, (a 2 ) = q, (a 2 
we arrive at 



0, {a?) = 0, (a 2 C 



2f2\ 



(1) 



1 



P(*I0= ( s ? + m^)5{a 2 - 1) + (1 - s £ )6(a), 

q — n 2 q — an 

S(= s - -£ , s ~ 



(12) 



1 — a 1 — a 

At this point we see from (|l^) that besides m and q 
the following parameter 

1 N 

»&,i = ^£k«.*l a ie?l a ( 13 ) 



2 



will play an independent role in the mutual informa- 
tion function. This quantity is called the activity-overlap 
since it determines the overlap between the active neu- 
rons, | (7i f | = 1, and the active parts of the memorized 
patterns, |£f | = 1. We remark that it also shows up in the 
alternative expression of the retrieval quality through the 
performance Pf — l/iVJ^ 5^ <cr . t (see PJL0|). It does 
not play any independent role in the time evolution of 
the network, independent of the architecture considered 
- diluted, layered or fully-connected. 

Next, one can verify that the probability p(cr|£) is con- 
sistent with the averages 



1 



™= -«cr) -|^>4 

1= «°" 2 > CT |«>6 



"=-«^ 2 ).i^ 2 )e- 

a ' 



(14) 
(15) 
(16) 



These averages are precisely equal in the limit N — > oo 
to the order parameters m and q in eq. (P|)-(R) and to 
the activity-overlap defined in eq. (|ll|) . (The fluctuations 
around their mean values can be neglected according to 
the LLN, hence the average over a particular i-site dis- 
tribution equals the infinite sum over i). 

Using the probability distribution of the memorized 
patterns (0) we furthermore obtain 



qS(a 2 -l) + (l-q)6(a). (17) 



The expressions for the entropies defined above then 
become 

5(ff) = -gln|-(l-g)ln(l-?), (18) 

(S{v\t)) e = aSa + (l-a)Si- a , (19) 

n + m n + m n — m n — m 

o„ = In In 

2 2 2 2 

-(1 -n)ln(l -n), (20) 
5i_« = -sln|-(l-a)ln(l-a) (21) 

and the mutual information is then given by eq. (^). We 
recall that mt, qt as well as nt are needed in order to 
completely know the mutual information content of the 
network at time t. 



IV. THRESHOLD DYNAMICS 

It is known that the parallel dynamics of fully con- 
nected networks is difficult to solve, even at zero temper- 
ature, because of the strong feedback correlations p| . 
Recently, a recursive dynamical scheme has been devel- 
oped which calculates the distribution of the local field 



at a general time step using signal-to-noise analysis tech- 
niques [l2] ]. Recursion relations are obtained determin- 
ing the full time evolution of the order parameters. We 
shortly review these results. 

Suppose that the initial configuration of the network 
{(Tifl}, is a collection of IIDRV with mean (cr^o) = 0, 
variance ((er^o) 2 ) = <kni an d correlated with only one 
stored pattern, say the first one {£,}}'■ 



N ^ 



01,0 



> 0. 



(22) 



This implies that by the law of large numbers (LLN) one 
gets for the retrieval overlap and the activity at t = 

' ' ' (23) 

go = Jim q N . Q = {al ) = q in . (24) 



lim rnjy o 

N— »oo 



lim q N . Q = (af Q ) = q m 



At a given time step of the dynamics, the state of a neu- 
ron, <Jij+i is determined by its local field at the previous 
time step. In general, in the limit N — > 00 the distri- 
bution of the local field at time t + 1 consists out of a 
discrete part and a normally distributed part fll2[j 



h i<t = £m\ + ^aaD t M(Q, 1) + 5 M 
D t = Var [rjfl = Var 



lim — 7=YAi a i- 



t'=0 



n * 



Oi.t' 



(25) 

,v>\ (26) 
(27) 



with A/"(0, 1) a Gaussian random variable with mean zero 
and variance 1 and \ the susceptibility 



Xt 



V aaD 



L= (^J Vz zF 9t (^rnj + y/aaDt z 



(28) 



where V is the Gaussian measure. In the above ((•)) 
denotes the average both over the distribution of the 
embedded patterns {£f } and the initial configurations 
{difi}. The average over the initial configurations is hid- 
den in an average over the local field through the updat- 
ing rule (Q). 

The first term on the r.h.s. of ( |25| ) is the signal term 
produced by the pattern that is being retrieved, the rest 
represents the noise induced by the (p— 1) non-condensed 
patterns. In particular, the second term is Gaussian noise 
and the last term Bit contains discrete noise coming from 
the feedback correlations. The quantity D t satisfies the 
recursion relation 
qt+i 



D t +i = 



+ xtDt + 2 Xt Cov[AA(0, qt+i/a), rf] (29) 



For more details we refer to |T^] . Using the above scheme 
the order parameters at a general time step can then be 
obtained in the limit TV — > 00 from Eqs. and 



3 



q t+ i = ((Fl(h i , t ))). 



(30) 
(31) 



The activity overlap needed in order to find the mutual 
information can also be written as 



1 



{mrKiKt)))- 



(32) 



Of course, we then also need to specify its initial value 

1 



nl= lim ^.^-((^Ko) 2 ) 



(33) 



The idea of the self-control threshold dynamics intro- 
duced in the diluted model Q and studied for some other 
models without feedback correlations |4|||f7j has been 
precisely to let the network counter the noise term in the 
local field at each step of the dynamics by introducing 
the following form 



% = c 



{a)\J aaDt, 



(34) 



where the function c(a) is a function of the pattern ac- 
tivity. We remark that in these cases without feedback 
there is no discrete noise in the local field (the term B i t 
in eq. ( ]25| ) is absent). Furthermore, also the covariance 
term in eq. ( p9| ) is absent. Moreover, for the diluted 
model D t — qt/a, i.e., only the first term in eq. (^9|) is 
present. So this dynamical threshold has two important 
characteristics. First, it is a macroscopic parameter hav- 
ing the same value for every neuron thus no average must 
be taken over the microscopic random variables at each 
time step. Secondly, it changes each time step but no 
statistical history intervenes in this proces. 

We see that the choice ( |34j ) is in fact related to the 
variance of the local field, taken for a fixed realization 
of the pattern which is being retrieved. It is the width 
of the noise produced by the non-condensed patterns. 
It is obvious that it cannot be taken to be a function 
of the overlap with the pattern being retrieved. As a 
consequence, for the fully connected network we cannot 
work with the exact form for Dt as given in eq. (|2^) be- 
cause of the presence of the covariance. So, if we want 
to take into account some effects of feedback correlations 
and if we want the threshold to have the characteris- 
tic properties mentioned above, we need to approximate 
the covariance term in eq. ( p9| ) such that only the pre- 
vious time step is involved. This is realised by approxi- 
mating this term by 2xt{Var[A/"(0, gt + i/a)]Var[r^ 1 ]} 1 / 2 = 



2x*fe+i/a] 1/2 [A] 1/2 . We then easily get 
aaDt+i = [G t + ^Jaqt+i] 2 

G t = (^Jvz zF 9t (^m] + ^foinD 



(35) 
(36) 



Furthermore, we take both contributions at equal times 
and call ^/aaD t = A t = 



on this approximation of the feedback correlations we re- 
fer to |l2|,[l3| and references therein. Finally, since Gt 
is a function of the overlap m\, a quantity which is not 
available to the network we replace it by Go = W2/w a. 

What is left then is to find a form for c(a). For the 
low-activity networks considered up to now the stor- 
age capacity could be considerably improved by taking 
c ( a ) = [— 21n(a)] 1 / 2 such that for the diluted model 
9 t = [— 2 ln(a)agt] -1 / 2 . The same form has been shown 
to work for the layered model Q| . For the fully connected 
model considered here we again propose, a priori, this 
form. So, combining these results we take as self-control 
threshold 



d t = V-21n(a) A° 



A 



(37) 
(38) 



Finally, we make one more assumption on the dynam- 
ics. In the local field distribution ( |25| ) we forget about the 
discrete noise B^t and suppose that the noise produced 
by the non-condensed patterns is Gaussian distributed. 
Computer simulations have shown that this assumption 
is approximately valid as long as the retrieval is succes- 
ful @. As a consequence we can write down recursion 
relations for the order parameters 



n t +i = J T>zFg t (ml + zA t ) 
qt+i =a J Vz[F et {m\ + zA t )} 2 
+(l-o) J Vz[F 0t (zA t )f 



(39) 



(40) 



with 



A t = ^Jaqi + a J Vz z Fe t (ml + zA t ) 

+(l-a) J VzzF et (zA t ) (41) 

where we have already averaged over £. We remark that 
the form of the last equation is different from the cor- 
responding equation for the diluted and the layered ver- 
sions of this model [j7|JT5| because of the feedback. 

The expressions for the overlap m t +i, the neural activ- 
ity qt+i and the noise A t due to the non-condensed pat- 
terns describe the (approximate) macro-dynamics of the 
fully-connected neural network. Besides the self-control 
model with the threshold given by eq. (0)-@) we also 
consider the model with the threshold fixed at its zero 
time value, i.e., 6 t = 8q. 

At this point we remark that when studying the mutual 
information, we want to introduce explicitly the activity- 
overlap parameter, nt (recall eq. (p"3|)) in the dynamics 
leading to the following expression for q t +i 



For more details 



qt+i = ant+i + (1 - a) s t+ i 



(42) 
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where St+i is then, obviously, defined by the integral of 
[Fg t (zA t )] 2 . This parameter s is precisely that intro- 
duced in Eq. ( |l2"| ) and measures the number of active 
neurons in inactive condensed pattern sites. 

In the following section we compare the retrieval prop- 
erties of a fully connected network governed by this ap- 
proximate dynamics with and without self-control with 
numerical simulations. The main aim is to show that self- 
control also works in the case of fully connected models. 



V. RESULTS 

A. Numerical Results 

We have solved the time evolution of our threshold 
dynamics with a time-dependent self-control threshold 
given by eq. (p7|)-(|38|)) and with a time- independent 
threshold where the neural activity is fixed by qt = qa- 
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FIG. 1. The evolution of the overlap mt for several initial 
values mo, with qo — 0.01 = a and a = 2 for the self-control 
model (left) and the fixed-threshold model (right). The dashed 
curves are a guide to the eye. 

We have studied the behavior of these networks in the 
range of pattern activities 0.01 < a < 0.67 i.e., from 
low-activities to a uniform distribution of patterns. 

For both thresholds it turned out that the best results 
were obtained by taking c(a) = y— 2 ln(a) + K , with 
K = for a > 0.1 and K = 0.5 for a < 0.1. At this point 
we remark, however, that the pure log form for c(a) is 
derived in the theoretical limit a — > 0. So, it may be that 
we did not reach small enough values in our numerical 
analysis (which is due to numerical complexity). We re- 
call that one of the main aims of this work is to show 
that self-control also works for fully connected models. 
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FIG. 2. The basin of attraction for the retrieval over- 
lap mo as a function of a for a = 0.01 and initial qo = a 
and no = 1 for the self-control model (full line) and the fixed 
threshold model (dashed line). 

The important features of self-control are illustrated 
in Figs. 1-5. In Fig. 1 we compare the time evolution of 
the retrieval overlap, m t , starting from several initial val- 
ues, mo, for the model with self-control, 6 SC = Qt (recall 
eq. ©-(H)) with the model with fixed threshold 9 . An 
initial neural activity qa = a = 0.01 and a loading a = 2 
have been taken. We observe that the self-control forces 
more of the overlap trajectories to go to the retrieval at- 
tractor m = 1. Only an initial overlap m ~ 0.4 for the 
self-control model versus m ~ 0.6 for the fixed threshold 
model is needed. We remark that for mo < 0.6 the over- 
lap decreases in the first time step for both models. This 
is an expression of the fact that correlations are especially 
important in the first time steps leading to a decreasing 
neural activity qt, but the self-control threshold is able 
to counter these effects. Near the attractor correlations 
seem to become less important and the Gaussian charac- 
ter of the local field distribution dominates. 

Since the initial overlap needed to retrieve a pattern is 
smaller for the self-control model, the basins of attraction 
of the patterns are substantially larger. This is further il- 
lustrated in Figs. 2-4 where the basin of atraction for the 
whole retrieval phase R is shown for both models with 
an initial value q = a = 0.01. We have calculated the 
fixed-point to^ of the dynamics (|3^), (|4(]) and ( f4l| ) and 
we have determined the intial conditions of the relevant 
parameters such that the network is able to retrieve, i.e., 
such that TOqo ~ 1. It is interesting to also give no and/or 
so separately in order to see how the activity qt is built 
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up. 




1 2 



go = ariQ or, equivalently sq — 0. So we consider small 
qo running from 0.004 to 0.01. We observe the peculiar 
behavior that for the fixed-threshold network an initial 
no > to is needed, but even then still no retrieval is pos- 
sible for low storage a < 0.1. For the self-control model 
a much broader region of retrieval exists. Finally, the 
specific role of the parameter St is displayed in Fig. 4. 
We start from a maximal initial overlap too — 1 and take 
no = 1 meaning that for so = to so = 1, qt runs from 
0.01 to 1. It can be seen that especially when so is get- 
ting large the storage capacity of both models decreases 
quite drastically but again much less for the self-control 
model. 



FIG. 3. The basin of attraction for the activity-ovelap no 
as a function of a for a — 0.01 and initial qo — ano and 
mo = 0.4 for the self-control model (full line) and the fixed 
threshold model (dashed line). 

In Fig. 2 we have used qo = a, no = 1. The basin of 
attraction for the self-control model is larger, even near 
the border of critical storage. Hence the storage capacity 
itself is also bigger. 
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FIG. 4. The basin of attraction for the parameter so (see 
main text) as a function of a for a — 0.01 with initial mo — 1 
and no = 1 for the self-control model (full line) and the fixed 
threshold model (dashed line). 

Furthermore, a smaller initial activity-overlap no suf- 
fices to have retrieval as is seen in Fig. 3. There we start 
with initial conditions too = 0.4, i.e., the smallest initial 
overlap possible for a = 2 as we recall from Fig. 1, and 
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FIG. 5. The information i as a function of a with 
self-control for several values of a: results using the evolution 
equations. 

We conclude with the observation that self-control 
works in a large range of pattern activities a, as shown 
in Fig. 5. There the mutual information content i — 
J2iJ2f Jl ^/(^ z Jij) — a I 1S plotted as a function of the 
loading a on a logaritmic scale. We observe the slow 
increase of i as the activity a decreases, saturating at a 
value close to i ~ 0.3. This behavior is typical for low 
activity networks ■ 
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B. Simulations 



Simulations have been carried out for systems with 
N = 10 neurons. For every new stored pattern /i, we 
start our dynamics with a state a{ = £f , and calculate 
the order parameters m t , n t and the activity overlap qt, 
using the definitions (0), (|t]) and (|l3|). To avoid a very 
large computation time wc have stopped the dynamics 
after t < 5 time steps when no convergence was reached 
before. Then we have averaged over windows in the p- 
axis in order to obtain the mutual information i. The 
window size runs from Sp = 50 for a — 0.67 (where we 
have stored p — 10 3 patterns) up to 8p = 2 x 10 3 for 
a = 0.01 (where we have stored p — 5 x 10 4 patterns). 



0.05 - 




we do not see a big difference between the two models 
(self-control and fixed threshold). Only for larger values 
of a, self-control shows a little improvement. For the 
low-activity case, however, the main role of self-control 
on the neural activity is clearly noticed since q ~ a in 
that case while in the fixed-threshold model it is impos- 
sible to control q such that it stays in the neigborhood 
of a. As a conseqeunce the mutual information, e.g., is 
only about half of that for the model with self-control. 
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FIG. 6. The information i as a function of a with 
self-control for several values of a: simulations with N = 10 4 
neurons. 

The conditions on the LLN mentioned in Section IVA 
are approximately fullfillcd for such large networks, since 
the fluctuations (neglected in Eqs.([l4])-(|l6|)) are of order 
l/\/aN. However, for smaller activities a, this quan- 
tity may be not so small. It becomes crucial in the case 
a = 0.01, where this quantity is 0.1, such that the finite 
size effects get relevant. This implies a kind of cut-off 
in the information for the self-control model as seen in 
Fig. 6. However, the agreement with the analytic results 
of Fig. 5 is quite good up to a — 0.01. 

In order to further understand the details of the re- 
trieval quality we plot in Fig. 7 all the parameters m, n, q 
for the model with and without self-control in two cases: 
a = 0.01, a low-activity case, and a = 0.67, implying a 
uniform distribution of patterns. For the uniform case 



FIG. 7. Simulations for N = 10 4 neurons of the infor- 
mation i, the overlap m, the neural activity q and the ac- 
tivity-overlap n as a function of a for the self-control model 
(filled circles) and the fixed threshold model (squares) after 
five time steps. The pattern activities are a = 0.01 (left) and 
a = 0.67 (right). 



Finally, in Fig. 8 we compare the simulations with the 
results from the fixed-points of the dynamics ( |39| ) , ( |40| ) 
and ( [ll| ) for a = 0.03. Up to t = 5 time steps are con- 
sidered for both the model with and without self-control 
and we have averaged over a window in the p-axis of 
size 5p — 10 3 . For the self-control model the small un- 
derestimation of the theoretical results can, of course, be 
attributed to the approximations of the noise term (recall 
Eqs @ and ©). 
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FIG. 8. The information i as a function of a with (thick 
full line and filled circles) and without (thick dashed line 
and squares) self-control for a = 0.03. Comparision between 
the results from the evolution equations and simulations with 
N = 10 4 neurons. 



VI. CONCLUDING REMARKS 

In this paper we have introduced a self-control thresh- 
old in the dynamics of fully connected networks with 
three-state neurons. This leads to a large improvement of 
the quality of retrieval of the network. The relevant quan- 
tity in order to study this, especially in the limit of low 
activity is the mutual information function. The mutual 
information content of the network as well as the critical 
capacity and the basins of attraction of the retrieval so- 
lutions for three-state patterns are shown to be larger be- 
cause of the self-control mechanism. Furthermore, since 
the mutual information saturates, the critical capacity of 
the low-activity network behaves as a c — 0(|aln(a)| _1 ). 
Numerical simulations confirm these results. 

This idea of self-control might be relevant for vari- 
ous dynamical systems, e.g., when trying to enlarge the 
basins of attraction and convergence times. Indeed, it 
has been shown to work also for both diluted and lay- 
ered networks. Binary as well as ternary neurons and 
patterns have been treated. In all cases, it turns out 



that in the low-activity regime the self-control threshold 
can be taken to be proportional to the square root of the 
neural activity of the network. 
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